Bayesian Nonparametric Inverse Reinforcement Learning

Inverse reinforcement learning (IRL) is the task of learning the reward function of a Markov Decision Process (MDP) given the transition function and a set of observed demonstrations in the form of state-action pairs. Current IRL algorithms attempt to find a single reward function which explains the entire observation set. In practice, this leads to a computationally-costly search over a large (typically infinite) space of complex reward functions. This paper proposes the notion that if the observations can be partitioned into smaller groups, a class of much simpler reward functions can be used to explain each group. The proposed method uses a Bayesian nonparametric mixture model to automatically partition the data and find a set of simple reward functions corresponding to each partition. The simple rewards are interpreted intuitively as subgoals, which can be used to predict actions or analyze which states are important to the demonstrator. Experimental results are given for simple examples showing comparable performance to other IRL algorithms in nominal situations. Moreover, the proposed method handles cyclic tasks (where the agent begins and ends in the same state) that would break existing algorithms without modification. Finally, the new algorithm has a fundamentally different structure than previous methods, making it more computationally efficient in a real-world learning scenario where the state space is large but the demonstration set is small

Review of the techniques used in motor‐cognitive human‐robot skill transfer

Abstract A conventional robot programming method extensively limits the reusability of skills in the developmental aspect. Engineers programme a robot in a targeted manner for the realisation of predefined skills. The low reusability of general‐purpose robot skills is mainly reflected in inability in novel and complex scenarios. Skill transfer aims to transfer human skills to general‐purpose manipulators or mobile robots to replicate human‐like behaviours. Skill transfer methods that are commonly used at present, such as learning from demonstrated (LfD) or imitation learning, endow the robot with the expert's low‐level motor and high‐level decision‐making ability, so that skills can be reproduced and generalised according to perceived context. The improvement of robot cognition usually relates to an improvement in the autonomous high‐level decision‐making ability. Based on the idea of establishing a generic or specialised robot skill library, robots are expected to autonomously reason about the needs for using skills and plan compound movements according to sensory input. In recent years, in this area, many successful studies have demonstrated their effectiveness. Herein, a detailed review is provided on the transferring techniques of skills, applications, advancements, and limitations, especially in the LfD. Future research directions are also suggested

Real-Time Human-Robot Interactive Coaching System with Full-Body Control Interface

Abstract. The ambitious goal being pursued by researchers participating in the RoboCup challenge [8] is to develop a team of autonomous humanoid robots that is capable of winning against a team of human soccer players. An important step in this direction is to actively utilise human coaching to improve the skills of robots at both tactical and strategic levels. In this paper we explore the hypothesis that embedding a human into a robot’s body and allowing the robot to learn tactical decisions by imitating the human coach can be more efficient than programming the robot explicitly. To enable this, we have developed a sophisticated HRI system that allows a human to interact with, coach and control an Aldebaran Nao robot through the use of a motion capture suit, portable computing devices (iPhone and iPad), and a head mounted display (which allows the human controller to experience the robot’s visual perceptions of the world). This paper describes the HRI-Coaching system we have developed, detailing the underlying technologies and lessons learned from using it to control the robot. The system in its current stages shows high potential for human-robot coaching, but requires further calibration and development to allow a robot to learn by imitating the human coach.

Raising AI: Tutoring Matters

Abstract. Humans and other animals are often touted as examples of systems that possess general intelligence. However, rarely if ever do they achieve high levels of intelligence and autonomy on their own: they are raised by parents and caregivers in a society with peers and seniors, who serve as teachers and examples. Current methods for developing artificial learning systems typically do not account for this. This paper gives a taxonomy of the main methods for raising / educating naturally intelligent systems and provides examples for how these might be applied to artificial systems. The methods are heuristic rewarding, decomposition, simplification, situation selection, teleoperation, demonstration, coaching, explanation, and cooperation. We argue that such tutoring methods that provide assistance in the learning process can be expected to have great benefits when properly applied to certain kinds of artificial systems